Click any tag below to further narrow down your results
Links
Randy Au argues that LLMs can take over tedious coding work in data analysis, but exploratory data analysis still needs a human who understands the data's context and intent. He says current models make poor analytical decisions and that the act of poking at data is how analysts sharpen their hypotheses.
- Exploratory analysis depends on intent: the same numbers look "correct" or "weird" depending on what you plan to do, and an LLM has no access to that purpose unless you spell out a huge amount of business context.
- Coding agents lack a feedback signal for analytical quality, so they confidently run off on flawed approaches. Au says he had to micromanage them to the minute detail, and the promised analysis often didn't match what was delivered.
- Hypothesis generation happens through hands-on iteration with the data. Offloading that work skips the process where questions get refined and interesting correlations get noticed.
OpenAI’s data productivity team built Kepler, an AI agent that understands their data platform context and writes iterative SQL to answer complex data questions. It handles table discovery, query refinement, follow-up prompts and even visualizations, cutting analysis time and reducing errors.
- OpenAI built Kepler, an internal AI agent that writes iterative SQL to answer data questions across 70,000 datasets and 600 petabytes daily, deployed via Slack, IDEs, or web UI
- It doesn't just query once—it discovers tables, refines CTEs, verifies numbers against dashboards/Notion, and proposes hypotheses (e.g., duplicate logging) for anomalies like a WAU spike
- Memory for self-learning plus rigorous evals let it improve over time without regressing
- Rolled out to 80% of the company, turning multi-engineer, multi-hour investigations into minutes